Robust power series algorithm for epistemic uncertainty propagation in Markov chain models

In this article, we develop a new methodology for integrating epistemic uncertainties into the computation of performance measures of Markov chain models. We developed a power series algorithm that allows for combining perturbation analysis and uncertainty analysis in a joint framework. We characterize statistically several performance measures, given that distribution of the model parameter expressing the uncertainty about the exact parameter value is known. The technical part of the article provides convergence result, bounds for the remainder term of the power series, and bounds for the validity region of the approximation. In the algorithmic part of the article, an efficient implementation of the power series algorithm for propagating epistemic uncertainty in queueing models with breakdowns and repairs is discussed. Several numerical examples are presented to illustrate the performance of the proposed algorithm and are compared with the corresponding Monte Carlo simulations ones

Efficient updating of node importance in dynamic real-life networks

The analysis of real-life networks, such as the internet, biometrical networks, and social networks, is challenged by the constantly changing structure of these networks. Typically, such networks consist of multiple weakly connected subcomponents and efficiently updating the importance of network nodes, as captured by the ergodic projector of a random walk on these networks, is a challenging task. In this paper, new approximations are introduced that allow to efficiently update the ergodic projector of Markov multi-chains. Properties such as convergence and error bounds for approximations are established. The numerical applicability is illustrated with a real-life social network example

Weak differentiability of product measures

In this paper, we study cost functions over a finite collection of random variables. For these types of models, a calculus of differentiation is developed that allows us to obtain a closed-form expression for derivatives where "differentiation" has to be understood in the weak sense. The technique for proving the results is new and establishes an interesting link between functional analysis and gradient estimation. The key contribution of this paper is a product rule of weak differentiation. In addition, a product rule of weak analyticity is presented that allows for Taylor series approximations of finite products measures. In particular, from characteristics of the individual probability measures, a lower bound (i.e., domain of convergence) can be established for the set of parameter values for which the Taylor series converges to the true value. Applications of our theory to the ruin problem from insurance mathematics and to stochastic activity networks arising in project evaluation review techniques are provided. © 2010 INFORMS

Series Expansions for Finite-State Markov Chains

This paper provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm

Naïve learning in social networks with random communication

We study social learning in a social network setting where agents receive independent noisy signals about the truth. Agents naïvely update beliefs by repeatedly taking weighted averages of neighbors’ opinions. The weights are fixed in the sense of representing average frequency and intensity of social interaction. However, the way people communicate is random such that agents do not update their belief in exactly the same way at every point in time. Our findings, based on Theorem 1, Corollary 1 and simulated examples, suggest the following. Even if the social network does not privilege any agent in terms of influence, a large society almost always fails to converge to the truth. We conclude that wisdom of crowds seems an illusive concept and bares the danger of mistaking consensus for truth

From Data to Stochastic Modeling and Decision Making:What Can We Do Better?

In the past decades we have witnessed a paradigm-shift from scarcity of data to abundance of data. Big data and data analytics have fundamentally reshaped many areas including operations research. In this paper, we discuss how to integrate data with the model-based analysis in a controlled way. Specifically, we consider techniques to quantify input uncertainty and the decision making under input uncertainty. Numerical experiments demonstrate that different ways in decision making may lead to significantly different outcomes in a maintenance problem

Maximum likelihood estimation by monte carlo simulation:Toward data-driven stochastic modeling

We propose a gradient-based simulated maximum likelihood estimation to estimate unknown parameters in a stochastic model without assuming that the likelihood function of the observations is available in closed form. A key element is to develop Monte Carlo-based estimators for the density and its derivatives for the output process, using only knowledge about the dynamics of the model. We present the theory of these estimators and demonstrate how our approach can handle various types of model structures. We also support our findings and illustrate the merits of our approach with numerical results

Measure-valued differentiation for stationary Markov chains

http://staff.feweb.vu.nl/bheidergot

An approximation approach for the deviation matrix of continuous-time Markov processes with application to Markov decision theory

We present an update formula that allows the expression of the deviation matrix of a continuous-time Markov process with denumerable state space having generator matrix Q* through a continuous-time Markov process with generator matrix Q. We show that under suitable stability conditions the algorithm converges at a geometric rate. By applying the concept to three different examples, namely, the M/M/1 queue with vacations, the M/G/1 queue, and a tandem network, we illustrate the broad applicability of our approach. For a problem in admission control, we apply our approximation algorithm toMarkov decision theory for computing the optimal control policy. Numerical examples are presented to highlight the efficiency of the proposed algorithm. © 2010 INFORMS